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ABSTRACT 

This thesis examines the deterioration of ship handling proficiency of Surface 
Warfare Officers on shore duty. A Surface Warfare Officer (SWO) develops ship 
handling proficiency during his or her first and second ship tours, then spends two or 
more years ashore. Upon returning to sea duty, an officer is expected to be proficient in 
ship handling even though it has been two years since the last shipboard evolution. 

Ashore SWOs were tested to determine whether their ship handling skills or knowledge 
about navigation rules had degraded over time. During the first phase of the experiment, 
subjects were immersed in a virtual environment to assess procedural knowledge of a 
ship handling task. The second phase of the experiment, designed to measure declarative 
knowledge of ship handling, consisted of a short written test. The results of the 
experiment showed no deterioration of SWOs ship handling skills over time. The results 
did show a significant deterioration of declarative knowledge of navigation rules. Actual 
or potential applications of this research include revising current Surface Warfare Officer 
training programs to account for the fact that not all knowledge is lost to memory equally. 
Periodic refresher training for SWOs on shore duty is also suggested by these results. 
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EXECUTIVE SUMMARY 

This thesis examined the degradation of ship handling proficiency of Surface 
Warfare Officers on shore duty. After acquiring and routinely exercising ship handling 
skills during his or her first two at-sea tours, a Surface Warfare Officer typically rotates 
to an ashore duty assignment for two or more years. During this time, the officer’s ship 
handling proficiency is not exercised, despite the fact that the officer will eventually 
return to sea and be expected to retain his or her previous skill level. 

To assess the extent of ship handling proficiency degradation, a computer 
simulation of an underway replenishment and a written exam were created and 
administered to a number of Surface Warfare Officers currently assigned to the Naval 
Postgraduate School. 

The experiment was divided into two phases, the computer simulation phase and 
the written exam phase. The purpose of the computer simulation phase was to assess the 
subject’s ship handling skill when performing a routine, but demanding, ship handling 
task. During this phase, the subject was immersed in a virtual environment simulating the 
bridge of a ship via a Virtual Research V8 head-mounted active matrix display. The 
subject’s task was to approach a replenishment ship, assume the proper station relative to 
that ship, and maintain station until the completion of the simulation. The subject 
maneuvered the approaching ship via verbal commands. Each subject performed one 
practice session to assess the maneuvering characteristics of the approaching ship. The 
practice session was followed by six trials, three each of two different simulated weather 
conditions. The computer simulation phase was followed by the written exam phase. 
The purpose of the written exam phase was to assess the subject’s declarative knowledge 
of ship handling fundamentals and U.S. Coast Guard Rules of the Road. The written 
exam was divided into two parts, a multiple-choice section and an identification section. 
The multiple-choice section covered ship handling fundamentals in a written format, 
while the identification section presented various lighting and day-shape configurations 
for the subject to identify. 
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Results suggest that procedural knowledge of ship handling, as measured by a 
computer simulation, are robust to deterioration over time. Conversely, declarative 
knowledge, as measured by a written exam, presented statistically significant evidence of 
decline over time. 

These results suggest ship handling training for SWOs returning to sea duty after 
shore duty should be weighted to address knowledge more heavily if it is more 
susceptible to decay. 



xii 



ACKNOWLEDGMENTS 



Thanks to Gretchen, for being so busy herself she could not possibly interfere 
with writing a thesis, except to make hot coffee and give encouragement. 

Thanks to Jake or Jasper (I don’t know for sure), who tried to include in this 
document four entire pages of the single letter “Q”. 

Thanks to Prof. Krebs for never giving up on a student who does not always score 
touchdowns and to Prof. Lucas for making sense of ugly data. 

Thanks especially to LCDR Tom Evanoff, for patience and for letting me try to 
catch his shadow as he raced to great things. 



xiii 



THIS PAGE INTENTIONALLY LEFT BLANK 



xiv 



I. 



INTRODUCTION 



A. RESEARCH PROBLEM 

Piloting a surface vessel is a skill that requires many years of practice to master, if 
indeed one can ever master such a complex interaction of motion and forces. Like other 
human skills, “ship handling proficiency” is not a single, identifiable piece of knowledge, 
such as one’s home phone number. Rather, it is a combination of facts learned in a 
classroom — the term “bow” meaning the front of a ship, for instance — and skills learned 
through experience — such as determining the right moment to reverse engines when 
docking. 

Learning to be a good driver is a useful analogy. A good driver must know the 
traffic laws, such as speed limits, the meaning of a red light, and so on. The driver must 
also understand the function of the steering wheel, brake pedal, and all of the other 
controls used to operate an automobile. A prospective driver will leam the rules of the 
road and receive hours of practice prior to the driver license examination. However, to 
be a good driver, rather than merely a licensed driver, one must gain experience driving 
an automobile on the road. 

Two kinds of knowledge contribute to the mastery of a skill. Memorizing and 
consciously recalling the significance of a red traffic light, for example, is declarative 
knowledge. However, steering a car out of an icy skid involves a different type of 
processing, one that, through practice, becomes an automatic, unconscious process. This 
is procedural knowledge (Willingham, Nissen & Bullemer, 1989). In the context of ship 
handling proficiency, one may leam the basic physical forces that cause a ship to move 
(i.e., declarative knowledge) before ever setting foot on a ship. A new surface officer 
will leam about wind, current, propellers, rudders, tugs and ground tackle in a classroom. 
However, it will take that officer many hours of experience to become proficient at 
handling a ship (i.e., procedural knowledge). 
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After acquiring a basic level of proficiency in ship handling, a Surface Warfare 
Officer (SWO) may not practice those skills for a long time. The normal career path of a 
SWO includes periods during which he or she will not exercise ship-handling knowledge. 
If officers lose skills during these periods, they must recover them later on at some cost to 
the navy. In addition, it is important to examine whether declarative knowledge and 
procedural knowledge decline at the same rate. 

B. BACKGROUND 

1. Declarative and Procedural Knowledge 

Modem psychology has come to view human knowledge as a combination of two 
distinct memorial, or knowledge, systems. These two systems are commonly referred to 
as “declarative knowledge” and “procedural knowledge” (or, more recently, “non- 
declarative knowledge”). Earlier terms for the two systems of memory included 
“automatic” and “controlled,” a distinction that perhaps more clearly conveys the 
hypothesized difference between the two systems. 

In general, procedural knowledge is thought to be a system that operates without 
conscious effort (Squire & Cohen, 1984). Examples of this type of knowledge include 
the semantic and syntactic rules of the language a person uses and the skill of driving. 
People speaking and listening in their native language intuitively know and practice 
correct semantic and syntactic rules (i.e., good grammar), but are in most cases unable to 
articulate them (Lewicki, 1986). They unconsciously access rules of grammar to produce 
proper speech or raise attention to speech that does not adhere to these rules. 

Procedural memory is also thought to support the acquisition and retention of 
skilled performance (Willingham, Nissen & Bullemer, 1989). Driving is a good example 
of this manifestation of procedural knowledge. A skilled driver recalls, without 
conscious effort, the procedural knowledge of a task such as shifting the manual 
transmission of a car into a different gear. The driver may, in fact, not even be aware that 
he or she has performed the task. 






A number of studies have suggested that one can, in fact, acquire procedural 
knowledge without consciously knowing that one has even learned anything. For 
example, one study exposed subjects to a sequence of frames containing a target and 
required them to search for the target in each frame (Lewicki, Czyzewska, & Hoffman, 
1987). Unbeknownst to the subjects, the location of the target in the seventh trial was 
predictable, based on the sequences of target location in four out of the six previous trials. 
Although none of the subjects indicated knowledge of the hidden pattern in post- 
experiment interviews, their performance on the seventh trial indicated that a significant 
level of learning took place. In essence, when the seventh trial commenced, the subjects' 
procedural knowledge acquired over the last six trials unconsciously cued them on where 
to look for the target. 

Declarative knowledge, in contrast to procedural knowledge, is consciously 
retrieved when needed. Declarative memory is thought to support the retention of facts 
and the recollection of prior events. Examples of declarative memory include 
remembering what you ate for dinner last night, or being able to describe the appearance 
of your car. One can communicate this type of information explicitly to other 
individuals, who can then form a fairly accurate mental model of the same information in 
their declarative memory. Procedural memory, however, is impossible to pass on to 
another individual. For example, one can verbalize declarative aids to the skill of riding a 
bicycle, such as “pedal” and “hold the handlebars,” but one can not verbalize the skill 
itself to another individual. 

There are other differences between the two types of knowledge. In particular, 
procedural memory is believed to be fairly specific to the original learning situation and 
not easily accessible by the declarative system (Squire, 1994). In other words, to recall 
an example already discussed, the procedural knowledge required to unconsciously 
operate a manual transmission on a car is very specific to the task of shifting a manual 
transmission. Declarative memory, in contrast, allows the flexible use of knowledge in 
situations different from the original learning context. For example, one could adapt 
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declarative knowledge of the mechanics of an automobile engine to a wide variety of 
similar, but different, devices. 

In addition to demonstrating differing mechanisms for declarative and procedural 
knowledge, studies have demonstrated an actual neurological basis for a distinction 
between the two memory systems. For instance, certain forms of brain lesions impair 
declarative, but not procedural, learning (Squire, 1986). A number of other studies 
focusing on amnesiacs support the theory that the two memory systems are, in fact, 
distinct. For example, in one study, amnesiac patients who have severely impaired 
declarative memory learned a classification task at the same rate as normal subjects 
(Reber, Knowlton, & Squire, 1996). However, the amnesiacs were impaired on transfer 
tests that required flexible use of their task knowledge. In other words, the amnesiacs 
learned the procedural aspect of the task as well as normal subjects, but were impaired in 
declarative knowledge of the task. 

Numerous studies on amnesia, other neurological disorders, and aging have 
revealed that procedural knowledge and declarative knowledge are, in fact, distinct. The 
two memory systems are not equally robust and are not stored or processed in the same 
way by the human brain. It appears from studies that declarative knowledge is the more 
fragile of the two forms of memory, while procedural knowledge is more robust, but less 
flexible. 

2. Acquisition and Utilization of Ship-handling Skills 

The acquisition and retention of ship-handling skills is central to the training of a 
new ensign in the surface community. At the Surface Warfare Officer School in 
Newport, R.I., practiced instructors teach new ensigns the fundamentals of ship handling 
in a classroom setting, supplemented by simulator training. Only upon arriving at their 
first ship, however, do aspiring surface warfare officers obtain most of their knowledge. 
They will typically get frequent opportunities to learn and practice ship-handling skills 
under the watchful eyes of more-experienced officers. Newly reporting surface officers 
will generally stand watch as the Conning Officer, while a more senior officer will 
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supervise in the role of Officer of the Deck. The Conning Officer is the person who 
directly controls the movement of the ship with verbal commands to the personnel 
operating the ship’s helm (steering control) and throttle (engine control). New officers, 
during their first shipboard tour, will spend many hours as Conning Officer and likely 
become proficient in a variety of ship-handling tasks, including very difficult tasks such 
as underway replenishment and plane guard. The goal of all new surface officers on their 
first ship is to become qualified as Surface Warfare Officers (SWO). 

The SWO qualification is analogous to an aviator’s “wings” or a submariner’s 
“dolphins.” To attain their SWO qualification, officers must satisfactorily perform a 
variety of ship-handling tasks and demonstrate before a board of senior officers a 
comprehensive grasp of essential surface warfare knowledge. Toward this goal, new 
officers are given many opportunities to exercise their ship-handling skills and spend 
many hours on the bridge. However, once they have earned their SWO qualification, 
usually on their first ship, officers may have far fewer opportunities to exercise their ship- 
handling skills. Though officers may still regularly stand watches on the bridge, they 
have less incentive to rigorously practice ship-handling skills, and they will most often 
act as the supervisory Officer of the Deck rather than the Conning Officer. In many 
cases, SWO qualified officers do not spend any time at all on the bridge because their 
watch stations involve some other aspect of the ship’s operation, such as engineering. In 
general, officers who have obtained the SWO qualification will seldom directly practice 
ship handling as Conning Officer. 

Following an initial at-sea period on two different ships, officers return to shore 
duty, typically for about two years. Prior to returning to sea duty after the first ashore 
period, officers will normally attend Department Head School in Newport, Rhode Island. 
Although Department Head School makes some effort to refresh ship-handling skills 
through both classroom instruction and simulator exercises, this training is neither as 
extensive nor as demanding as the initial at-sea qualification process. In total, officers can 
expect about 20 hours of ship-handling refresher training while at the school. The 
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inclusion of refresher ship-handling training implies an assumption that the ship-handling 
skills of SWOs deteriorate while on shore duty. 

It is true that, after shore duty, officers may return to a ship not having stood a 
bridge watch for at least two years and, in most cases, longer than that. If, after SWO 
qualification, officers stood most of their watches somewhere besides the bridge, it could 
be as long as four or five years since they practiced ship-handling skills in a significant 
manner. Nevertheless, upon arrival at their first ship after shore duty, officers will be 
expected to be expert ship handlers. The ship-handling refresher training at Department 
Head School is an attempt to address this issue. However, the structure of the refresher 
training, with its emphasis on simulator exercise over classroom instruction, implies a 
pattern of knowledge deterioration that may not be accurate. And, SWOs on shore duty 
are not routinely tested to determine whether or not such deterioration actually occurs. In 
fact, it may be the case that declarative knowledge (e.g., “bow means the front of the 
ship”) degenerates at a different rate than procedural knowledge (e.g., “when to reverse 
the engines when docking”), which is learned by experience. 

The ratio of simulator to classroom training in the ship-handling refresher phase 
of Department Head School is 16 to 1. As mentioned earlier, classroom training imparts 
primarily declarative knowledge. And, although ship-handling simulator training 
assumes competency in appropriate declarative knowledge, it exercises primarily 
procedural knowledge. The overwhelming emphasis on simulator exercises implies an 
assumption that declarative knowledge is much more robust to deterioration than is 
procedural knowledge. This assumption may not be correct. 

Military training of many varieties, not least the extensive training that goes into 
creating a proficient Surface Warfare Officer, is expensive in terms of both time and 
money. If the skills of SWOs deteriorate during periods of inactivity, that represents a 
very real cost to the navy. Not only has the navy lost some of the value of its initial 
investment in its officers, but it also must expend additional time and money to retrain the 
officers before they return to sea duty. Therefore, if refresher training is indeed justified 
by a measurable decline in skills, it would be prudent to selectively tailor that refresher 
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training to target those skills that have declined. Other communities within the navy and 
other branches of the armed services face similar dilemmas, and this line of research may 
benefit their training programs, as well. 

3. Underway Replenishment 

The specific ship-handling task modeled in the computer simulation phase of the 
experiment is underway replenishment (UNREP). This particular task was chosen 
because it is one of the most challenging ship-handling tasks a Surface Warfare Officer 
can expect to encounter on a routine basis. It is also one of the tasks at which SWOs 
must become proficient in order to achieve their qualification. 

The main purpose of UNREP is to replenish fuel and stores at sea. The 
replenishing of vessels while underway, which essentially eliminates the need to return to 
port, is one of the cornerstones of the U.S. Navy’s ability to operate globally. UNREP is 
particularly important when ships must make long transits or remain on station for 
extended periods. 

Replenishment at sea is normally accomplished by means of fuel hoses suspended 
from steel cables that are under tension between a supply ship and another vessel. The 
system of steel cable, tensioning equipment and connection point on each ship from 
which a fuel hose is suspended is called a rig. Rigs can be used to transfer, in addition to 
fuel, pallets of food, parts, ammunition or other supplies, and underway replenishment 
has become a common evolution for ships at sea. 

Replenishment at sea does carry risk, however: ships must steam very close to one 
another for extended periods; constant, minute adjustments must be made in both course 
and speed to maintain proper separation between the two ships; and equipment failure, 
miscommunication, inattention or poor ship handling can easily cause a collision. 
Because of these dangers, replenishment at sea is a delicate and demanding ship-handling 
task, although years of refining procedures have made serious accidents rare. 
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For this experiment, UNREP is used as a representative ship-handling task to 
assess subjects’ procedural knowledge of ship handling. Because of the specificity of 
procedural memory discussed above, it is possible that subjects can perform well on the 
UNREP task, but not as well on other ship-handling tasks. However, as UNREP 
encompasses many of the individual skills an officer must acquire in order to be 
proficient at ship handling, UNREP indeed appears to represent the procedural aspects of 
ship handling as a whole. 

C. PURPOSE 

The purpose of this thesis is to determine whether Surface Warfare Officers’ ship- 
handling skills and knowledge deteriorate over time. The hypothesis is that SWOs’ ship- 
handling abilities will deteriorate at a rate different from that of maritime declarative 
knowledge. The experiment consists of a mixed-design model with time away from sea 
serving as the between-subject variable. 

It was assumed that there would be a high degree of correlation, for Surface 
Warfare Officers, between the amount of time since standing bridge watches while on sea 
duty and the current quarter of instruction at the Naval Postgraduate School (NPS). It was 
thought that by assigning subjects to one of two groups based on current quarter of 
instruction (i.e., first quarter students as one group, seventh quarter students as another), 
each group would be homogenous in terms of time away from bridge watch at sea. 

Two different lighting conditions and two different orderings of the computer 
simulation trials served as the within-subject independent variables. The dependent 
variables measured were distance from correct station (in meters) for the computer 
simulation trials and number of questions answered correctly for the written test. The 
data collected were used to determine if the dependent variables changed significantly as 
a function of the independent variables. 

It was expected that procedural knowledge, as measured by the computer 
simulation, would be robust to deterioration over time and, conversely, that declarative 
knowledge, as measured by the written exam, would be much less robust over time. The 
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distinction between the two types of ship-handling knowledge should, therefore, manifest 
itself as a difference in the two groups’ performance on the written exam measuring 
declarative knowledge, but no significant difference between the groups on the computer 
simulation measuring procedural knowledge. 
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II. METHODS 



A. SUBJECTS 

Eleven military observers with a mean age of 30 years and sigma of three years 
with normal or corrected to normal vision volunteered for this experiment. All subjects. 
Naval Postgraduate School graduate students, were designated as Surface Warfare 
Officers (SWO). The subjects were divided into two groups: first quarter students or 
graduating seventh quarter students. In order for a subject to participate in this 
experiment, he or she must have performed at least one UNREP and acted as a conning 
officer or officer of the deck prior to arriving at the Naval Postgraduate School. Informed 
consent was obtained from all observers. 

B. APPARATUS 

1. Virtual Environment Generation 

Subjects viewed the virtual environment through a Virtual Research V8 head- 
mounted active matrix LCD VGA display with a field-of-view of approximately 60 
degrees. Head positions were tracked with a 3space Polhumus tracking system and the 
ship position was manipulated by a BG Systems FlyBox joystick. The virtual 
environment was rendered on a Silicon Graphics Onyx Reality Engine. Software used to 
model the simulation was MultiGen Creator (version 14.5), Vega (version 3.2), Vega 
Marine module (version 3.2), and LynX graphical user interface (version 3.2) from 
MultiGen-Paradigm Inc. 

2. Flag Configuration for Computer Simulation 

During a normal daylight underway replenishment, the conning officer of the 
receiving vessel determines the lateral distance to the supply vessel by use of a visual aid 
called a phone and distance line. The phone and distance line, termed as such due to its 
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dual purpose as a visual aid and communication line, is shot between the vessels, via a 
reduced charge rifle cartridge, once the receiving vessel is approximately in station 
alongside. Colored flags are attached to the phone and distance line every five feet (1.5 
meters). The colored flags follow a repeating pattern of red, yellow, blue, white and 
green [Figure 1], 




Figure 1: During an UNREP, colored flags (red, yellow, blue, white, and green) are 
used as a visual aid to estimate the distance to the supply ship. 

On an actual phone and distance line, the distance in feet from the receiving ship 
to the supply ship is also painted on each flag. For example, the first three flags counting 
away from the receiving ship are a red flag with the number 5, a yellow flag with the 
number 10, and a blue flag with the number 15. The flags of the phone and distance line 
connecting the supply ship and receiving ship in the simulation did not have numerals on 
them. 
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The normal distance between the supply ship and receiving ship varies with the 
sea-state, type of ships involved, and the weather. In most cases, the two ships are 
approximately 90 feet to 120 feet apart (Crenshaw, 1975), though ships can extend to 
approximately 300 feet apart before refueling hoses are in danger of disconnecting. For 
purposes of the experiment, the correct lateral separation was defined at 80 feet due to the 
HMD’s narrow field-of-view. The subjects were made aware of the correct lateral 
separation for the experiment. 

C. PROCEDURE 

1. Design of Experiment 

The experiment was divided into two phases, procedural phase and declarative 
phase. During the procedural phase subjects were immersed in a virtual environment and 
assumed the responsibilities of a conning officer of a cruiser. As conning officer, the 
subject was required to perform six underway replenishments with a simulated supply 
vessel. The six trials were divided into three daytime and three dusk UNREP 
simulations. During the daytime simulation, the supply ship maintained a speed of 13 
knots. The dusk condition illumination was reduced by 30 percent and the supply ship 
altered speed in a random range between 13.8 and 14.4 knots to simulate the effects of a 
higher sea-state. The six trials were randomly ordered for each subject. Each simulated 
UNREP was six minutes in length. 

At the completion of the procedural phase, the subject was given a two-minute 
rest before the start of the declarative phase. The declarative phase was a written exam 
that lasted approximately 20 minutes. Refer to Appendix A to review the exam. 

Before the first procedural trial commenced, subjects were briefed on the ship 
handling task of an underway replenishment. Once the subject was familiarized with the 
simulation task the experimental trials began. The subject controlled the cruiser’s course 
and speed by issuing verbal commands to the test monitor. The test monitor, in turn, 
acted as helmsman and lee helmsman of the simulated cruiser, controlling its course and 

speed via a FlyBox controller. Within the simulation, the output of the FlyBox was 
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adjusted to accurately mimic the handling characteristics of a Ticonderoga-class cruiser 
in terms of speed and acceleration, turning radius, advance and transfer. 

Before beginning the first trial, subjects were allowed one practice run six 
minutes in length. The beginning position of each vessel for each trial is depicted in 
Figure 2. Figure 3 depicts the subject’s view through the HMD of an approach in 
progress. The purpose of the practice run was to familiarize the subject with the handling 
characteristics of the simulated receiving vessel, the appearance of the simulated supply 
vessel, and to practice verbal commands. No data collection was performed during the 
practice run and subjects were free to ask questions. 



r 




Receiving Vessel 



Supply Vessel 




475 meters 



C7I 




Figure 2: Starting positions of the receiving ship and the supply ship. 



During the dusk lighting condition trials, the supply ship continually altered speed 
in a range between 13.8 and 14.4 knots. The simulated supply ship followed a slightly 
twisting path that brought her course alternately to the left and right of 000° [Figure 4], 

The supply ship and cruiser position and heading were recorded every five 
seconds. The speed of the supply ship and cruiser were displayed on a monitor every five 
seconds. 
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Differences between the day and dusk conditions were as follows: 

• Ambient Light Level: The light level for the dusk condition was reduced 
approximately 30% compared to the day condition. 

• Supply Ship Speed: During the day condition trials, the supply ship maintained a 
speed of 13 knots. During the dusk condition trials, the supply ship altered speed in a 
range between 13.8 and 14.4 knots to simulate the effects of a higher sea-state. 
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Figure 4: Supply ship track. 

The declarative phase assessed subjects’ knowledge of ship handling 
fundamentals and the International Rules of the Road. During this phase of the 
experiment, the subject was given a multiple choice and short answer written exam 
consisting of 18 questions [Appendix A], Of the 18 questions, eight were multiple choice 
questions concerning ship handling fundamentals and regulations as described in Naval 
Shiphandling and the International Regulations for Prevention of Collisions at Sea, or 
'‘Rules of the Road”. The remaining ten questions asked the subject to visual identify the 
type of, or task being performed by, vessels depicted in a series of illustrations. Each 
illustration presented the day shapes or running lights of a type of vessel typically 
encountered while underway. 



16 



The subject was allowed 20 minutes alone in a room to complete the exam. The 
subjects were not allowed to use references or a calculator. The experiment concluded 
upon the subject’s completion of the written exam. 
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III. RESULTS 



A. SEPARATION OF GROUPS FOR ANALYSIS 

A comparison of current quarter of instruction and time away from bridge watch 
at sea for the subjects of the experiment revealed that, in line with expectations, students 
near the end of their time at NPS had spent at least a year and half away from sea. 
However, students near the beginning of their instruction at NPS exhibited a wide range 
of time, between one and 18 months, away from sea. There are many reasons for this 
phenomenon, including ship schedules, billet assigned while on sea duty, and personal 
circumstances. 

Thus, experimental group one represents subjects who had, at the time of the 
experiment, spent anywhere between one to 18 months away from bridge watch on sea 
duty. Experimental group two represents those who had spent more than 18 months 
away from bridge watch on sea duty. 

B. PHASE I : SIMULATED UNDERWAY REPLENISHMENT 

1. Measures of Performance 

The computer simulation phase of the experiment consisted of 1 1 subjects 
conducting six simulated underway replenishments each, for a total of 66 observations. 
Subjects were assessed on three separate measures of performance: 

• Time to Station : The amount of time, measured in seconds, from the 
beginning of a trial until the subject maneuvered the receiving vessel to within 
100 meters of the target station. 

• Overall Error : Mean difference in position between the receiving vessel and 
the target station over the duration of an entire trial. 
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• Post-Approach Error : Mean difference in position between the receiving 
vessel and the target station over the portion of each trial subsequent to the 
receiving vessel maneuvering within 100 meters of the target station. 

The distinction between overall error and post-approach error was made to 
account for differing styles of ship maneuver. The environment in which Surface 
Warfare Officers learn how to maneuver into station influences how they will conduct an 
evolution such as UNREP. Some commanding officers advocate a fast, aggressive 
approach at high speed, while others prefer a slower, more deliberate approach. By 
making a distinction between overall error and post-approach error, the environment in 
which the subjects acquired their ship handling skills will not confound an assessment of 
their performance. Furthermore, making a distinction between overall error and post- 
approach error allows an analysis of whether or not there is a systematic difference 
between groups in time to reach station. 

2. Statistical Analysis 

Statistical analysis of the data was done using S+ 2000 software from MathSoft, 
Inc., and Arc 0.98 software (Cook and Weisberg, 1999). 

An initial examination of the time to station data using normal probability and 
histogram plots suggested that the data were not normal [Figure 5], A Kolmogorov- 
Smimov goodness-of-fit test supported the hypothesis that the time-to-station data did not 
follow a normal distribution. As a result of the initial analysis, a Box-Cox power 
transformation (time 067 ) was performed to make the data more amiable to multi-factor 
analysis of variance (MANOVA) [Figure 6]. 

Similarly, an initial analysis of the overall error data suggested that these data did 
not adhere to a normal distribution [Figure 7]. As with the time-to-station data, a power 
transformation (error' ' ) was performed to reshape the data distribution to a more 
symmetrical form [Figure 8]. 
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An analysis of post-approach error (overall error corrected for approach time) 
indicated that the data did not follow a normal distribution [Figure 9]. A power 
transformation was performed on the post-approach error data to make them more 
symmetrical forMANOVA [Figure 10]. 





Normal quantiles 

Figure 5: Histogram and probability plot of time to station data. 





Tine Normal quantiles 

Figure 6: Histogram and probability plot of time to station data after 

transformation. 
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Figure 7: Histogram and probability plot of overall error data. 




Overall Error 




Normal quantiles 



Figure 8: Histogram and probability plot of overall error data after 

transformation. 



22 



o 




Normal quantiles 



Figure 9: Histogram and probability plot of post-approach error data. 




Post Approach Error 



to 




Normal quantiles 



Figure 10: Histogram and probability plot of post-approach error data after 

transformation. 
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A two-factor MANOVA did not reveal significant effects due to the two lighting 
conditions (Pillai-Bartlett Trace F ( i, 62 )= 0.104, g = 0.08) or two subject groups (Pillai- 
Bartlett Trace F ( i, 62 )= 0.0106, p = 0.89). The interaction term lightingrsubjects was also 
not significant (Pillai-Bartlett Trace F ( i, 62 )= 0.009, g = 0.90). 

3. Observations 

The results of the computer simulation phase of the experiment were suggestive, 
but were not statistically significant evidence of a decline in procedural knowledge over 
time. Both groups statistically performed equally well overall, with no significant 
differences in any measure of performance for either condition. Group 1 performed 
slightly better than Group 2 in overall stationing error and post-approach error under both 
lighting conditions, with the exception of overall error in the day condition [Figure 11]. 



Mean O veratl S tatio n in g Error 




Figure 11: Comparison between groups for overall error. 
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Mean Post-Approach Stationing Error 




Figure 12: Comparison between groups for post-approach error. 



The small difference in performance between subject groups was most apparent in 
the post-approach error measure for the dusk condition [Figure 12]. This small difference 
may be construed as evidence of a very slow deterioration in procedural memory over 
time, but the results of the statistical analysis do not support that hypothesis. 

There was also a small difference between the two groups for the time-to-station 
dependent variable. On average, subjects in Group 1 took about ten seconds longer than 
subjects in Group 2 to reach station in the day lighting condition. In the dusk condition, 
subjects in both groups took, on average, the same amount of time. 
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4 . 



Ship Track Data 



During the computer simulation phase of the experiment, the positions and 
courses of the two ships were recorded every five seconds. A plot of these data for each 
trial provides an overhead visual representation of the simulated UNREP. Figure 13 
depicts one trial. 



Supply Ship and Cruiser Tracks during UNREP 
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Figure 13: Example of position data collected during each trial. 
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Figure 14 depicts another useful plot of the position data collected during each 
trial. The plot shows the distance from the cruiser to the correct station and depicts the 
same UNREP as Figure 13. In this particular trial, the subject approached the correct 
station rapidly during the first 200 seconds. For the remainder of the trial, the subject 
attempted to reach and maintain station with small course and speed adjustments. Near 
the end of the trial the subject nearly achieved perfect station, as can be seen by the dip in 
the data almost to the X-axis after time 300. 



Distance to Station 




Time (seconds) 



Figure 14: Plot of distance to correct station for UNREP depicted in figure 13. 
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c. 



PHASE II: WRITTEN EXAM 



1. Measures of Performance 

The written exam consisted of 18 questions. Of these, eight were multiple choice 
questions concerning ship handling fundamentals and regulations as described in Naval 
Shiphandling and the International Regulations for Prevention of Collisions at Sea, or 
Rules of the Road. The remaining ten questions asked the subject to visually identify the 
type of vessel, or the task being performed by the vessel, in a series of illustrations. Each 
illustration presented the day shapes or running lights of a type of vessel typically 
encountered while underway. 

For the written exam portion of the experiment, subjects were assessed on the 
following measures of performance: 

• Incorrect answers on Fundamentals portion : The number of incorrect answers 
on the first eight questions of the exam. The questions on this portion of the 
exam were multiple-choice. Possible scores were zero to eight. 

• Incorrect answers on the Rules of the Road portion : The number of incorrect 
answers on the Rules of the Road portion of the exam. Each question on this 
portion of the exam required two written answers and was, therefore, scored 
as two points each. Possible scores were zero to 20. 

A distinction between the two portions of the exam was made because in each 
portion both the aspect of declarative ship handling knowledge tested and the method of 
testing that knowledge differed. Questions in the first portion of the exam were based on 
written information found in Naval Shiphandling and the International Regulations for 
Prevention of Collisions at Sea, or “Rules of the Road.” Below each question were four 
possible answers. The subjects were required to circle the answer they thought was 
correct. 
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In contrast, each question on the second portion of the exam presented visual 
representation of the day shapes or running lights of a type of vessel typically 
encountered while underway. Subjects were required to visually identify the type of 
vessel, or the task being performed by the vessel, in each illustration. Subjects were also 
required to identify the target angle of the vessel depicted. Target angle is similar to 
aspect, or orientation, and refers to the relative headings of each vessel. Target angle is 
an important concept in the Rules of the Road since it is often the single factor used to 
determine right-of-way when vessels encounter one another at sea. 

2. Statistical Analysis 

An initial examination of the data seemed to indicate two trends [Figure 13]. 

First, scores on the fundamental portion of the exam did not seem to provide evidence of 
a difference between groups (solid line in Figure 15). On the other hand, scores on the 
Rules of the Road portion of the exam seemed to strongly indicate a difference between 
groups (dashed line in Figure 15). 




1 2 3 4 5 6 7 8 9 10 11 



Sublet 

Fund ES2 IZ3 R or — - Linear (Ror) L mear (Fund) 



Figure 15: Questions missed on each portion of the exam ( with trendlines). 
Subjects 1-5 are Group 1, Subjects 6-11 are Group 2. 
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G ro up 2 



Figure 16: Between group comparison of exam scores. 



Clearly, a comparison of the group means for the two portions of the exam 
illustrates the dichotomous results. For the Fundamentals portion of the exam, the group 
means are almost identical (group 1 mean of 3, group 2 mean of 3.16). For the Rules of 
the Road portion of the exam, the group means appear widely separated (group 1 mean of 
5.2. group 2 mean of 1 1). 

A number of methods were applied to the data to determine whether the 
difference suggested by the visual assessment was, in fact, significant. 
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A t-test showed no significance for fundamentals (t = -0.1903, df = 9, g = 0.8533), 
as was expected. A t-test for Rules of the Road did show statistical significance at the 
a=.05 level, (t = -2.5125, df = 9, p = 0.0332). 

A Wilcoxon rank-sum test revealed no significance for the fundamentals portion 
of the exam, as was expected (Z = -0.1882, p = 0.8507). A Wilcoxon rank-sum test also 
did not reveal significance for the Rules of the Road portion of the exam (Z = -1.9437, 
p = 0.0519). 

3. Randomization Test 

Conflicting results between the t-test and Wilcoxon rank-sum test, as well as the 
nature of the data, suggested a distribution-free non-parametric approach at this stage. As 
a result, a randomization test was performed. 

A randomization test is a form of permutation test based on randomization. The 
test is carried out by computing a test statistic for the experimental data, then randomly 
permuting the experimental data repeatedly and computing the test statistic from the 
permuted data each time (Kotz, Johnson, and Read, 1986). A p-value is obtained from 
the proportion of permutations that result in a computed test statistic as great or greater 
than the test statistic computed from the experimental data. The null hypothesis of a 
randomization test is that the measurement for a subject is independent of group 
assignment. The p-value obtained from a randomization test is thus the probability that, 
assuming the null hypothesis is true, one would observe a test statistic as extreme as that 
observed in the experiment. An important characteristic of a randomization test is that it 
is entirely independent of any assumptions about the distribution of the data under 
scrutiny. 

For the exam data, the null hypothesis is that a subject’s score on each portion of 
the exam is independent of what group that subject is in. To test this assumption free of 
assumptions concerning the distribution of the exam data, a randomization test was 
performed using 100,000 permutations for each portion of the exam. The randomization 
test showed no significance for fundamentals (g = 0.89), as was expected. For the Rules 
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of the Road portion of the exam data, the randomization test did show statistical 
significance (p = 0.047) at the a=0.05 level. 
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IV. DISCUSSION 



A. SUMMARY OF RESULTS 

This research suggests Surface Warfare Officers experience a degradation of 
declarative knowledge of ship handling while away from sea. On the other hand, no 
evidence was found to reject the hypothesis that procedural knowledge of ship handling 
remains intact while away from sea. 

The results of the simulated UNREP portion of the experiment revealed no 
significant differences between groups or lighting conditions. The results provide no 
evidence to support the assumption that procedural knowledge of ship handling 
deteriorates over time. 

The results of the written exam portion of the experiment revealed a significant 
difference between two groups distinguished by time away from sea. On the visual 
identification portion of the exam, scores for Group 1 were significantly better than the 
scores of Group 2. No difference between groups was detected on the multiple-choice 
portion of the exam. 

B. IMPLICATIONS 

The results of this experiment indicate that, as expected, SWOs experience 
deterioration of ship handling proficiency while away from sea. However, the 
deterioration does not equally affect both procedural knowledge and declarative 
knowledge. Declarative knowledge was shown to have deteriorated while away from sea 
but procedural knowledge was not. The ship handling refresher training at Department 
Head School is directed primarily at procedural knowledge. The results of this 
experiment suggest that more time should be spent refreshing declarative knowledge 
instead. 

Specifically, time away from sea was shown to have a significant effect on the 
ability of the subjects to visually identify signals displayed in accordance with the 
International Regulations for the Prevention of Collisions at Sea. Frequently, the signals 
described in these regulations provide the only common language between ships on the 
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high seas. Rapid and correct identification of a ship’s signals can be vital to avoiding a 
collision or grounding. For instance, misidentifying the lights of a vessel aground (as a 
number of subjects did) could cause a ship to hit the same shoal itself. On the exam, 
every light and day shape question depicted just such a potentially dangerous situation. 

Correct identification of lighting configurations is not only a matter of safety for 
one’s own vessel. Some lighting configurations indicate that a vessel is in trouble and in 
need of assistance. For example, there is a lighting configuration for ships “not under 
command”. Not under command means that the ship is uncontrollable, usually due to 
mechanical breakdown. A ship may be adrift for many days in this condition before 
encountering a potential rescuer. Food and water supplies may run low on such a ship 
and medical problems may arise among passengers and crew. Misidentification of the 
“not under command” lights could cause a potential rescue ship to steam past a disabled 
vessel without rendering assistance or notifying authorities. 

Light signals are also used an aid to determine the relative heading of another ship 
(i.e. coming towards you, going away, on parallel course, etc). The relative heading of 
another ship usually determines whether it has the right-of-way. Although radar has 
greatly improved the ability to determine a ship’s course, collisions still occur as a result 
of confusion about the right-of-way. 

The small amount of time spent to refresh SWOs on the Rules of the Road may 
not be enough to offset the significant deterioration demonstrated by this research. If so, 
SWOs are returning to sea without the necessary declarative knowledge to stand bridge 
watch safely. When SWOs return to sea they are expected to train the junior SWOs, but 
officers may be standing bridge watches without the knowledge to effectively carry out 
that task. 

There is no reason to believe the decay in knowledge is unique to NPS students. 
The large population of SWOs on shore duty likely exhibit the same deterioration 
detected in subjects at NPS. Additionally, the distinguishing factor between groups was 
not “time away from sea” per se, but rather “time since bridge watch”. Consequently, 
there may be many SWOs at sea who experience decay in declarative knowledge of the 
Rules of the Road because they are not standing bridge watches. 
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c. 



FOLLOW-ON RESEARCH 



There are a number of potential areas of follow-on research in this topic. For 
example, further research could attempt to detect changes in declarative and procedural 
knowledge of ship handling over smaller time spans. 

Ships periodically remain in port for repair or refit for six months or more. 
During these periods, the SWOs on the ship are not exercising their ship handling skills. 
Furthermore, during these periods SWOs on the ship are heavily burdened with tasks 
related to repair and maintenance and do not have time to refresh ship handling skills. 
Follow-on research, with a greater number of subjects, could determine whether these 
SWOs are also at risk of decay in declarative ship handling knowledge over small time 
spans. 

Another area of follow-on research could examine different ship handling tasks. 
Underway replenishment may be fundamentally different from other ship handling tasks. 
A repeat of the same experiment using a different task, such as plane guard or pier 
landing, might yield different results. A combination of two or more ship handling tasks 
could also be used. Subjects may exhibit decay in procedural knowledge of some tasks, 
but not others. 

Follow-on research could also examine other areas of proficiency. For example, 
SWOs typically are required to achieve advanced shipboard fire-fighting qualifications in 
order to lead fire-fighting efforts in the event of a fire. Fire-fighting skills may also be in 
danger of decay during routine periods of inactivity. 

Another interesting question is whether the results found by this experiment are 
also found in other similar populations. Subjects might be military pilots or vehicle 
drivers who spend significant periods away from their primary duty, for example. 

The computer simulation created for this experiment could also be used for 
research in other topics. In particular, the Tactical Vectoring Equipment (TVE), a lighted 
visual navigation aid developed at NPS by LCDR Tom Evanoff (Evanoff, 1999; Krebs, 
Evanoff, and Sinai, 2000) was evaluated using a very similar computer simulation. The 
computer simulation developed for this experiment would provide an excellent tool for 
evaluating the effectiveness of the TVE as an aid to approach and station keeping for 
UNREP. 
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D. RECOMMENDATIONS 



The results of this research suggest refresher training for SWOs should be revised. 
Before returning to sea after extended periods ashore, SWOs should be retrained in the 
declarative aspect of ship handling at least as much as the procedural aspect. 

This research does not suggest ship handling refresher training should not address 
procedural tasks. Simulator training in an accurate and high fidelity simulator can be of 
benefit, even when the skills exercised are intact beforehand. Simulator training can be 
used to practice rare or dangerous tasks, or to gain familiarity with new procedures or 
equipment. 

This research does suggest declarative knowledge of ship handling is susceptible 
to decay. Importantly, this decay occurs within a time period SWOs spend ashore as a 
normal part of their careers (often more than once). Decay of declarative knowledge 
should be expected during these periods. Refresher training should be weighted to 
address knowledge more heavily if it is more susceptible to decay. Specifically, refresher 
training for prospective department heads, executive officers and commanding officers 
should devote more time to the Rules of the Road. 
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APPENDIX A. WRITTEN EXAM 



( correct answers are in italics ) 

SHIPHANDLING KNOWLEDGE QUIZ FOR SHIPHANDLING SKILL RETENTION THESIS 



Date of Test: Subject No.: 



(Assigned during testing) 



You have 20 minutes to complete the quiz. Circle the letter corresponding to one answer 
only for each multiple-choice question. 

Section One: Naval Shiphandling Fundamentals 



1. According to Crenshaw’s Naval Shiphandling , which of the following are the six main sources of force 
affecting the motion of a ship in the water independent of any other vessel? 

a) Current, Wind, Weather, Tide, Propellers, Tugs 

b) Current, Wind, Weather, Propellers, Rudders, Mooring Lines 

c) Current , Wind , Propellers, Rudders ; Mooring Lines , Ground Tackle 

d) Current, Wind, Propellers, Tugs, Rudders, Mooring Line 

2. If a 600-foot vessel seen “beam-on” subtends an arc of 2° at the observer, what is the range of the 
vessel from the observer? 

a) 1,500 yards 

b) 3,000 yards 

c) 4,500 yards 

d) 6, 000 yards 

3. What is the course of a contact that bears 045° true and has a target angle of 225°? 

a) 045° 

b) 135° 

c) 225° 

d) 000° 

4. Your vessel is directly astern an aircraft carrier at a range of 36,000 yards. Formation speed is 13 
knots and stationing speed is 30 knots. You are ordered to assume station 2,000 yards astern the 
carrier. Assuming no speed or course changes by the aircraft carrier, approximately how long will it 
take you to reach your new station? 

a) 20 minutes 

b) 45 minutes 

c) } hour 

d) 1 hour 20 minutes 
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5. What is the name of line 2 in the below figure of a vessel moored to a pier? 

a) After bow spring 

b) Forward quarter spring 

c) Forward bow spring 

d) After quarter spring 




6. As a rule of thumb, approximately how far from your initial course should you turn when beginning a 
Williamson Turn ? 

a) 30° 

b) 60° 

c) 75° 

d) 90° 

7. What is the sound signal while in fog for a vessel restricted in her ability to maneuver? 

a) 1 prolonged blast every two minutes 

b) 2 prolonged blasts every two minutes 

c) 1 prolonged blast followed by three short blasts every two minutes 

d) 1 prolonged blast followed by two short blasts every two minutes 

8. Nearly all single screw vessels, naval and civilian, have a tendency to do which of the following when 
backing? 

a) Veer to port when viewed from astern 

b) Veer to starboard when viewed from astern 

c) Veer one way, then the other 

d) None of the above 
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Section Two: Rules of the Road-Day Shapes 

The following questions are short answer. 

9. In the figure of the vessel below, what is the displayed signal indicatin 




Answer: Not under command 




Answer: Aground 



11. In the figure of the vessel below, what is the displayed signal indicating? 




Answer: Gear extended over the side (i.e. dredging , laying submarine cable , etc.) 



On what side of the vessel is it safe to pass? Side with diamonds 



Section Three: Rules of the Road-Lights 

The following figures show the lights of other vessels as seen directly ahead at a range of 1 nautical mile or 
less. All lights are being correctly displayed in accordance with the International Rules of the Road. 

12. In the figure of a vessel below, what is the displayed signal indicating and what is the approximate 
target angle (+/- 45°)? 



♦ 

* 



Answer: Tug and tow < 200 meters long ; TA =90° 
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13. In the figure of a vessel below, what is the displayed signal indicating and what is the approximate 
target angle (+/- 45°)? 




Answer: Tug pushing ahead; TA = 000 ° 



14. In the figure of a vessel below, what is the displayed signal indicating and what is the approximate 
target angle (+/- 45°)? 
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15. In the figure of a vessel below, what is the displayed signal indicating and what is the approximate 
target angle (+/- 45°)? 




Answer: Aground; TA ^90° 



16. In the figure of a vessel below, what is the displayed signal indicating and what is the approximate 
target angle (+/- 45°)? 
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17. In the figure of a vessel below, what is the displayed signal indicating and what is the approximate 
target angle (+/- 45°)? 




Answer: Not making way (or at anchor), obstruction on side with red lights ; TA = 000 °orTA = 180 ° 
(this may be bow or stem aspect since all lights displayed are 360° arc of visibility) 

18. In the figure of a vessel below, what is the displayed signal indicating and what is the approximate 
target angle (+/- 45°)? 
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APPENDIX B. SUBJECT QUESTIONNAIRE 

QUESTIONNAIRE FOR SHIPHANDLING SKILL RETENTION THESIS 



Date of Test: Subject No.: 



(Assigned during testing) 



Last Name: First Name: 

Rank: 

Age: Sex: Male / Female Citizenship: U.S. Foreign: 



Email Address: 

Military Service: Navy Marine Army Air Force USCG Other: Civilian 

Years of Active Military Service: 

Military Designator/Warfare Specialty: 

1. Is your vision at least 20/20 or correctable to 20/20? Yes /No 

2. Do you currently have any significant eye related health problems or significant 
problem 

with your night vision? Yes /No 

3. Provide a brief history of ships assigned and duties: 

Ship Name Hull Type/No Primary Duties Date 

BUNKER HILL CG 52 OPS(example) Aug 95-Dec 97 



4. How many months has it been since you last stood a bridge watch as OOD or 
Conning Officer? 

1-3 3-6 6-9 9-12 12-18 More than 18 

5. How many months has it been since you last conned a ship alongside during an 
UNREP? 

1-3 3-6 6-9 9-12 12-18 More than 18 
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APPENDIX C. SUBJECT BRIEF 

SUBJECT TEST PROCEDURES AND CHECKLIST 



• FORMS: Fill out questionnaire and consent form. Review for problems. 

• While subject fills out forms: 

• Compile subject files. 

1. Ensure move/rename previous subject “data” files. 

2. Type: orderb 

3. In ROOT, type: chmod +x+x+x (ba_(subject number and initials)) 

• Example (chmod +x+x+x ba_15abc) 

• Switch to HMD mode: 

1. In ROOT directory: 

2. Type: cd /usr/gfx 

3. Type: /setmon -S 4@640x480_60 

4. Type: ./stopgfx 

5. Turn on black box 

6. Type: ./startgfx 

• BRIEF: 

• Review UNREP, explain lack of hydrodynamic effects and narrow separation 
from supply ship. Explain correct station: 

1 . Fourth green flag just on cruiser rail. 

2. Phone-and-distance line perpendicular to both ships. 

• “In this simulation, you will be acting as the conning officer of a cruiser during 
six underway replenishments. Your ship will begin each UNREP in waiting 
station approximately 500 yards behind the supply ship. Your task is to make an 
approach and then maintain station on the supply ship. Each trial will begin with 
your ship at rest and the supply ship moving away from you at about 13.5 knots. 
When you are near the supply ship, the phone-and-distance line will automatically 
become visible.” 

• Review commands sheet. “I will be acting as the helmsman lee helmsman. 
Commands are come right, come left or steady. Engine commands are engines 
ahead, engines back or engines steady. Each trial will last 6 mins. There will be a 
short pause between each trial if you need to rest.” 

• TRIAL RUN: 

• “We will have one practice UNREP for you to get used to your ship and 
commands as well as to get a look at the supply ship. There is no phone-and- 
distance line in the practice UNREP.” 

• Post signs. Lights off. 

• Type: Bdatapull2 test7.adf 

• After trial, answer any questions. 
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• When ready for actual trials: 

• Type: ba_subject number and initials 

• (example: ba_15abc) 

• After last trial, turn on lights, let subject take a quick break. 



• BRIEF EXAM: 

• “You will now be given 20 minutes to complete a multiple choice and short answer 
quiz. No reference materials or calculators are allowed.” 

• Leave subject alone in separate room with quiz. 

• SAVE DATA FILE UNDER A NEW NAME. 

• Data will be in “data” file. Rename this file or else next subject data will 
overwrite it. 

To return HMD to normal mode, from ROOT: 

1. Type: cd/usr/gfx 

2. Type: ./setmon -x 1280xl024_60 

3. Type: ./stopgfx 

4. TURN OFF BLACK BOX 

5. Type: ./startgfx 
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